Resource-Aware Parallel Adaptive Computation for Clusters

Smaller institutions can now maintain local cluster computing environments to support research and teaching in high-performance scientific computation. Researchers can develop, test, and run software on the local cluster and move later to larger clusters and supercomputers at an appropriate time. This presents challenges in the development of software that can be run efficiently on a range of computing environments from the (often heterogeneous) local clusters to the larger clusters and supercomputers. Meanwhile, the clusters are also valuable teaching resources. We describe the use of a heterogeneous cluster at Williams College and its role in the development of software to support scientific computation in such environments, including two summer research projects completed by Williams undergraduates. Cluster computing environments at smaller institutions have provided a new platform for research and teaching in high-performance computing. Such local computing resources support development of software which can be executed on the local cluster or can be moved later to larger clusters or supercomputers for execution of larger problems. Meanwhile, clusters provide valuable local resources for teaching and the support of student projects. This paper describes a cluster at Williams College and provides an overview of a research effort that has been motivated and supported by this cluster, in particular two undergraduate projects which have contributed to this effort. 1 A Cluster Environment Our cluster (known as “Bullpen”1) is located in the Department of Computer Science at Williams College. It consists of one Sun Enterprise 220R server with one 450MHz Sparc UltraII processor; two Enterprise 420R servers, each with four 450MHz Sparc UltraII processors; and six Enterprise 220R servers, each with two 450MHz Sparc UltraII processors; and four Sun Ultra 10 Workstations, each with one 300 or 333 MHz Sparc UltraII processor. This cluster is intentionally heterogeneous, with its nodes having different processor speeds, numbers of processors and amount of memory per node. This makes it an excellent platform for studies of scientific computation in heterogeneous and hierarchical 1 http://bullpen.cs.williams.edu/ 2 J. D. Teresco, L. Effinger-Dean and A. Sharma environments. While most clusters are initially built using identical nodes, incremental growth is an attractive features of clusters. As new (likely faster) nodes are added, old nodes remain part of the cluster, leading to heterogeneity. In addition to the support of the research described herein, this cluster has been used in Computer Science courses at Williams, most extensively in the Parallel Processing course. Students have been able to write multithreaded code using both POSIX threads [5] and OpenMP2 to use the symmetric multiprocessing (SMP) nodes. They have used the Message Passing Interface (MPI)3 to use multiple nodes to perform parallel computation with distributed memory and message passing. Student projects have included a parallel discrete event simulation, parallel particle simulations, a parallel photon mapper and a parallel ray tracer. Having the local cluster available meant that the students were not competing for processor cycles on lab workstations and did not have to develop software remotely at a supercomputing center. 2 Parallel Adaptive Computation on Clusters Our goal is to develop tools and techniques to allow efficient parallel adaptive scientific computation on heterogeneous clusters such as Bullpen. We focus on solvers for systems of partial differential equations using finite element and related methods (e.g., [4, 6, 7]) that use meshes to discretize problem domains. The mesh is partitioned into subdomains consisting of disjoint subsets of mesh entities (e.g., elements, surfaces, nodes) and these subdomains are assigned to the cooperating processes of a parallel computation. Adjacent mesh entities will exchange information during the solution process. So in addition to its attempts to divide the work evenly among the processes (to achieve load balance), a mesh partitioner attempts to minimize the number of pairs of adjacent entities which are assigned to different processes (to minimize interprocess communication). The methods are adaptive, where time and space efficiency is improved by concentrating computational effort in parts of the domain where it is needed to achieve a solution to a prescribed accuracy [1]. However, adaptivity will disturb a balanced partitioning, necessitating a dynamic load balancing step. Dynamic load balancing procedures have similar goals to mesh partitioners, but must operate on already-distributed data and should minimize the change between the existing decomposition and the new decomposition (to limit mesh migration). A number of approaches to dynamic load balancing have been proposed ([8] includes a survey). The Zoltan Parallel Data Services Toolkit [2] provides a common interface to high-quality implementations of several such procedures. With Zoltan, applications can quickly make use of and can easily switch among available load balancing methods. Fig. 1 shows the interaction between parallel adaptive application software and a dynamic load balancing suite such as that in Zoltan. After an initial partitioning, the application performs computation steps, periodically evaluating error estimates and checking against specified error tolerances. If the error is within tolerance, the compu2 http://www.openmp.org 3 http://www-unix.mcs.anl.gov/mpi/ Resource-aware Parallel Adaptive Computation for Clusters 3 Load Balancing Suite Application Software